Papers with Transformer networks
Is Encoder-Decoder Redundant for Neural Machine Translation? (2022.aacl-main)
Copied to clipboard
| Challenge: | Encoder-decoder architecture is widely adopted for sequence-to-sequence modeling tasks. |
| Approach: | They propose to combine bilingual and multilingual translations to train a language model to do translation. |
| Outcome: | The proposed approach performs on par with the baseline encoder-decoder Transformer . the proposed approach is compared with the translation model in the target language . |
ETAS: Zero-Shot Transformer Architecture Search via Network Trainability and Expressivity (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing Transformer Architecture Search methods are limited to computer vision and natural language processing tasks. |
| Approach: | They propose a Transformer Architecture Search proxy that measures trainability and expressivity of Transformer networks separately and integrates it into an effective regularized evolution framework to demonstrate its efficacy. |
| Outcome: | The proposed proxy can achieve higher correlation with the true performance of Transformer networks on computer vision and natural language processing tasks. |
Multi-resolution Annotations for Emoji Prediction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Emojis are able to express various linguistic components, such as emotions, sentiments, events, etc. emojis have the merit of preserving information more densely, compared to words, argues a new study. |
| Approach: | They propose to use passage-level and aspect-level emoji annotations to predict the proper emmojis associated with text. |
| Outcome: | The proposed method is heuristically generated and validated with a pre-trained BERT model. |